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ABSTRACT. We develop a static complexity analysis for a higher-order functional language with 
structural list recursion. The complexity of an expression is a pair consisting of a cost and a potential. 
I— I ' The former is defined to be the size of the expression's evaluation derivation in a standard big-step 

Qh . operational semantics. The latter is a measure of the "future" cost of using the value of that expression, 

ly^ ■ A translation function ||-|| maps target expressions to complexities. Our main result is the following 

O ' Soundness Theorem: If t is a term in the target language, then the cost component of ||?|| is an 

upper bound on the cost of evaluating t. The proof of the Soundness Theorem is formalized in Coq, 
• providing certified upper bounds on the cost of any expression in the target language. 

>■ 

CO 

^ ■ 1. INTRODUCTION 

co 

\q | Though cost analyses are well-studied, they are traditionally performed by hand in a relatively 

ad-hoc manner. Formalisms for (par tially) autom a ting the analysis of higher-orde r fun ctional lan- 



guage s have been developed by, e.g. JShultis lll985tl . lSandsl lll990h . lVan Stonel i2003ll . and lBenzingeJ 



[2004]Q These formalisms map target-language programs into a domain of complexities, which can 



then be reasoned about more-or-less formally. The translations carry over information regarding the 
values of subexpressions, which can then be "discarded" during reasoning about cost should that be 



appropriate. 



In this paper we aim for a similar goal, but take an approach inspired by the work of Da nner and Rover 



[2009]. There we analyzed a programming language for type-level 2 functions in a restricted type 
system motivated by work in implicit computational complexity. The goal there was to prove that 
all programs in our formalism are computable in type-2 polynomial time. Here we take the same 
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analysis tools and apply them to a version of Godel's System T , not to establish fixed time bounds, 
but to construct expressions that bound the cost of the given program. The key difference between 
our approach and those discussed above is that we aim for upper bounds on cost in terms of input 
size, rather than an exact analysis in terms of values. This more modest goal allows us to develop 
a notion of complexity that bounds the run-time cost of any target program evaluation on inputs of 
a given size. Besides providing us with a simpler setting in which to reason about complexity, the 
absence of values in the upper bounds is also in line with typical algorithm analysis. A long-term 
goal is to incorporate well-established analysis techniques into the formal system of reasoning that 
we present here. 

In this paper we consider a higher-order language, defined in Section |2l over integers and 
integer lists with structural list recursion. Since the interesting analyses are done in terms of the 
sizes of the lists involved, we declare all integers to be of some constant size, which we denote by 1. 
As an example of our analysis, let us consider list insertion, defined by 

ins (x, nil) = [x] 

ins (x, y::ys) = if (x<=y) then x : : y : : ys else y::(ins(x, ys)) 

A simple cost analysis of ins yields a recursive cost function ins_c for which the recurrence 
argument is the length of the list argument of ins: 

ins_c(0) = Co ins_c(« + 1) = c\ + [cj V (cj + ins_c(n))) 

where the c, are constants. The maximum in the recursion clause ensures that we need not consider 
the value of the test x <= y. An easily-formalized proof by induction tells us that ins_c(«) € 0{n). 
But this is not enough for our purposes. We want our analyses to be compositional, so that we can 
directly use the analysis of, say, ins in the analysis of insertion-sort defined by 

ins_sort xs = fold ins xs nil 

An implicit part of the analysis of ins_sort uses the size of ins xs in terms of the size of xs. Thus 
in addition to the cost analysis above, our approach also generates a size analysis, which we refer to 
as potential (more on this terminology momentarily). The following is a possible potential analysis 
of ins: 

ins_p(0) = 1 ins_p(« + 1) = (n + 1) V (1 + ins_p(n)) 
where again we take a maximum so that we need not consider the value of the test x <= y. 

Since we are interested in higher-order languages, we also need to consider algorithms which 
take functions as input. For this, we need some way to represent the cost associated with using 
a function; this, combined with size, is our notion of potential, which we describe more fully in 
Section [3] For type-level objects, we can think of potential as ordinary size, which is why ins_p 
is a function from integers to integers. The potential of a function essentially encompasses its cost 
analysis — the potential of / is a map from potentials p (representing the potential of the argument) 
to the complexity of applying / to an argument of size p. Our complexity analysis of an expression 
yields a pair consisting of a cost (of computing that expression to obtain a value) and a potential 
(representing the size of that value). We refer to such a pair as a complexity. 

The complexity language that we define in Section [3] allows us to express recurrences like 
ins_c and ins_p. In Section|4]we define a translation function ||-|| from the target language to the 
complexity language so that ||ins|| encompasses these two recurrences. The recurrences that result 
from the translation will be more complicated than those shown here because they take complexities, 
as opposed to sizes, as arguments. But as shown in Section 14.1.11 it is easy to extract ins_c and 
ins_p from ||ins||. 

As another example, consider the map function defined by 
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map h nil = nil 

map h (x :: xs) = (h x) :: (map h xs) 

The natural cost analysis of map yields a recurrence that depends on the cost of applying h to 
account for the term h x in the recursion clause. Note that this is not the cost of h itself (or any 
specific function argument). Indeed, since any specific function argument is likely to be expressed 
as a A -abstraction, the cost of such an argument would be 1. Instead, we must refer to the potential 
of the function argument h; applied to the potential of a list element x (representing the size of x), we 
obtain the complexity of the application h(x). That complexity is a pair consisting of the cost of the 
application and its potential. Taking into account that our integers have constant size 1, we end up 
with a cost analysis that looks something like the following, where h now represents the complexity 
of the function aigumentjl 

map_c(/z,0) = map_c(/j,?i + 1) = (h p (l)) c +map_c(/z,n) 

where (-) c and (-) p extract the cost and potential components of a complexity, respectively. The 
potential analysis is straightforward and does not depend on the function argument: 

map_p(/j,0) =0 map_p(/j,?i + 1) = 1 +map_p(/j,«) 

It is nice that our translation gives the expected costs in these examples, but of course we want 
to know that it is sound, in the sense that ||f || bounds the complexity of t. We prove such a Soundness 
Theorem in Section |5]and state clearly the corollary that an upper bound on the evaluation of cost t 
can be derived directly from ||f ||. 

The function || • || is computable, and hence provides a formal link between the program source 
code and its complexity bound. We have implemented a subset of the target and complexity lan- 
guages, translation, and proof of the Soundness Theorem in Coq, and we discuss some of the details 
of this in Section [6] The formalization thus provides a mechanism for providing certified upper 
bounds on the complexity of target language expressions. This is in distinction to a more traditional 
ad-hoc analysis; such an analysis, even if formalized in a system such as Coq, is not tied to the 
source code in a machine-checkable manner, and so one still does not have a proof that the code to 
be executed satisfies the cost bounds that are asserted. 

2. Target language 

The target language is essentially a variant on Godel's System T; its syntax, typing, and operational 
semantics are given in Figures [Q-|3] The language provides for higher-order programming over 
integers, booleans, and integer lists (the base types). The latter are defined as a recursive datatype, 
and the datatype definition automatically provides for structural recursion. The work in this paper 
extends to other comparable recursive datatypes; we treat the special case here to cut down on 
notation]! Since the language is straightforward, we omit many of the details. A term is a typeable 
expression r h t : T. The operational semantics defines a big-step call-by-value evaluation relation 
that relates closures (r h t : to values v 6 (we usually drop the typing details and just write t E,). 
A closure t E, consists of a term t and a value environment E, that maps variables to values such 

As we will see, parameters in complexity functions that correspond to (list-) recursion arguments represent the 
potential of such arguments. Parameters that correspond to non-recursion arguments represent the complexity of those 
arguments. 

3 Our language does not support general recursion, and recursive datatypes in general pose some difficulties; we discuss 
both issues in Section[8] 
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that Dom £ contains all of t's free variables. We write {} for the empty environment. A value v8 
consists of a value expression and a value environment. A value expression is any of the following: 

• A boolean value tt orff or integer value n£Z; 

• Any finite sequence (no,... ,«/t-i) of integers; 

• Any expression of the form Xx.r. 

We often write (n,ns) for the sequence (n,no,. . . ,nk-i) when ns = (no, . . . ,nk-i) and write () for 
the empty sequence. Since the grammar does not allow lists of higher-order expressions and the 
semantics does not have side-effects, we can safely drop the environment at leaves of evaluation 
derivations that derive values of base type. We do so in order to simplify the statements of the rules 
for evaluating lists; it is not necessary in practice. To further clean up notation, we often write tt or 
ns instead of tt {} or ns{}. 

The evaluation of fold r of (s, [x,xs,w]t) when rt, \, (n,ns) deserves some comment. A more 
natural rule might be 

rt, \. (n,ns) (f oldnj of (s, [x,xs,w]t))£ 4- 00 t^i^vd 
(fold r of (s,[x,xs,w]t))t; |v0 
However, the fold "expression" in the hypothesis is not well-formed, because ns is not an expres- 
sion, it is a value. There is an obvious isomorphism between the two that we could employ, but 
then evaluating the hypothesis fold expression would require re-evaluating the list corresponding 
to ns which would add (possibly non-trivially) to the cost of the evaluation. Thus we choose a fresh 
variable y and bind y to ns in the environment; as a result, every future evaluation of the recursion 
argument will have cost 10 

We define the cost of a closure t%, cost(^), to be the size of the evaluation derivation of 
tt, (it is straightforward to prove that derivations are unique). We charge unit cost for arithmetic 
operations and count every inference rule. We can easily adapt the system for other notions of 
cost (e.g., counting only creation of cons-cells) by modifying the complexity semantics described 
in Section [3] 

3. The complexity language 

Our goal is to assign a complexity \\t || to each target language expression t. We have three desiderata 
on ||-||: 

• ||? || must provide an upper bound on the cost of evaluating t. 

• || • || must be compositional. 

• ||? || must not depend on the value of any subexpression of t. 

Since closures, not expressions, are evaluated in the target language, ||?|| will also be an expression, 
the meaning of which is determined by an environment assigning complexities to its free variables. 
To say that || • || is compositional means that ||?|| depends only on expressions for subexpressions s 
of t. Because of the third constraint, the information we provide about the evaluation cost of t is 
not as precise as that of as the systems mentioned in the introduction. However, this constraint is in 
line with almost all the work in practical analysis of algorithms and thus opens up the possibility of 
using the considerable body of tools and tricks of analysis of algorithms in our verifications. 

A complexity measure that considers only cost is insufficient if we want to be able to handle 
higher-order expressions like map. To see why, consider any expression Xx.r such that map(A^.r) 



I f fold were defin ed in terms of a more general letrec constructor, then a standard operational semantics of letrec 
as in | Reynolds, 1998] would introduce a comparable fresh variable. 



A STATIC COST ANALYSIS FOR A HIGHER-ORDER LANGUAGE 



5 



e € Exp ::= X | Z | true | false | nil | e :: e \ e R e \ 
if e then e else e \ Xx.e \ ee 
case e of (e, [x,xs]e) | fold e of (e, [x,xs,w]e) 

Figure 1. Target language grammar. X ranges over variable identifiers, Z over inte- 
ger constants, and R over numerical binary relations such as < and > and numerical 
binary operators such as + and x . 

a, T ::= int | bool | int* | a — > T 
int* ::= nil | :: of (int, int*) 

rheo^int* r h e\ : a r,x: int,xs: int* h ei". o 

r h case of (e\, [x,xs]e2) '■ O 
rheo^int* T\~e\:o F,x : int,xs : int*,w : o h e2 : 
r h fold eo of (£1, [jc,xs,w]e2) : G 

Figure 2. Target language types and typing. Rules not shown here are the expected ones. 



x$l$(x) c$lc{} (kx.r)£l(kx.r)Z 

r%ln r {} sE,\,n s {} n r Rn 



(rRs)^itt{} 
r£,jn r {} s^jn s {} ->{n r Rn 

(rRs)$ lff{} 
r^\, n r{\ s^-ln,*{} n = n r »n s 



s (R=<,<=,...) 



(R= <,<=,...) 



(• = +,-, x,...) 



rE, in{} s^lns{} 



nil£|(){} (r:: S )U(n,ns){} 
r%l(Xx.r Q )do s^jviOy r d Q {x ^ y l di} j vd 

r<§4(){} Ivd r^X(n,ns){} t £,{x,xs i-> n,ns} J. v0 



(case r of (j, [x,xy]f))£ ^ v (case r of (s, [x,xs]t))% IvG 

(foldrof (s,[x,xs,w]t))% ivd 
r^l(n,ns){} (f oldy of (s, [x,xs,w]t))%o i v % t^ilvd 
(foldrof (s,[x,xs,w]t))£ ivd 
where y is fresh; £o = £ {y ns}; £i = £ {x,xs, if4/i, ras, vo do} 

Figure 3. Target language operational semantics, c ranges over integer and 
boolean constants and c over the corresponding values. The third hypothesis of 
the relation and operation rules has unit cost. 
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is well-typedjf| Assuming that ||-|| is compositional, if ||/|| were to provide information on just the 
cost of evaluating t, then since the cost of Xx.r is 1, the cost of map(Ax.r) would be independent 
of r. What we need instead is a complexity measure such that ||/|| not only captures the cost of 
evaluating t, but also the cost of using t. We call this latter notion potential, and a complexity 
will be a pair consisting of a cost and a potential. To gain some intuition for the full definition, 
we consider the type-level and 1 cases. At type-level 0, the potential cost of an expression is a 
measure of the size of that expression's value. Now consider a type-level 1 expression r. The use 
of r is its application to a type-level expression s. The cost of such an application is the sum of 
(i) the cost of evaluating r to a value Xx.r'; (ii) the cost of evaluating s to a value v'; (iii) the cost of 
evaluating r'[x h-> v']; and (iv) a "charge" for the inference. Since (iii) depends (in part) on the size 
of v' (i.e., the potential of s), by compositionality complexities must capture both cost and potential. 
Furthermore, (iii) is defined in terms of the potential of r (i.e., the potential of Xx.r'). Thus the 
potential of a type-level 1 expression should be a map from type-level potentials to type-level 
complexities, and in general the potential of an expression of type a — > Z should be a map from 
potentials of type-a expressions to complexities of type-T expressions. 

We now turn to the formal definitions given in Figures We define a complexity language 
over a simple type structure with products into which our translation function ||-|| will map. The 
full complexity types consist of the simple types with products over N, which is intended to be the 
natural numbers and represents both costs and potentials of base-type values. The complexity and 
potential types are defined by the following mutual induction, following our preceding discussion 
about the potential of higher-order values: 

(1) If 7 is a potential type, then N x 7 is a complexity type. 

(2) N is a potential type. 

(3) If 7 is a potential type and z a complexity type, then 7— > z is a potential type. 

We introduce two notations for potential types 7: 7° = N x 7 and 7— >- n z = (7—)- t) d (remember 
that the potential of a function is a map from argument potentials to result complexities). 

The complexity expression constructors roughly mirror the target expression constructors, and 
the meaning of the former is intended to capture the complexity (cost and potential) of the latter. The 
grammar of complexity expressions is as expected for the product-related types, except we write t c 
and t p for the first and second projections (for "cost" and "potential"), respectively. The usual 
abstraction and application are replaced by \*x.r and r*s. As we will see, we need an operation 
that "applies" ||r|| to \\s\\; since both are complexities, and hence pairs, ordinary application does 
not suffice. Thus we introduce the * operator so that we can define ||rj|| = ||r|| * X*x.r is 
the corresponding abstraction operator. Conditional expressions are eliminated in the complexity 
language. A conditional expression is translated to (essentially) the maximum of the complexity 
of its two branches. This matches our interest in upper bounds on complexity, ensuring that the 
complexity of a conditional bounds the cost of any possible evaluation of that conditional on inputs 
of a given size or smaller. The fold constructor for recursive datatypes has a counterpart pf old 
in the complexity language so that recursive definitions in the target language are translated to 
recurrences in the complexity language. Instead of branching on a constructor, pf old branches 
potentials. Since case can be seen as a trivial version of fold, it has a corresponding counterpart 
pcaseQ 

"'We phrase this informal discussion in terms of evaluating closed expressions rather than closures. 
^In fact, we could also define conditionals in the target language to be syntactic sugar for a case over the trivial 
recursive datatype of booleans, and we would end up with the same complexity semantics. 
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Meanings are assigned to complexity terms (typeable complexity expressions) through a deno- 
tational semantics. We take the standard denotation [•] of full complexity types (interpreting N as 
the natural numbers). If T is a full complexity type environment, we say that ^ is Y-consistent if 
^(x) E [r(^)] for all x G Dom T. Finally, we define a function [-J— that maps a complexity term 
r h e : T and Inconsistent environment £, to [T h e : tJ £ G [f] . We write JeJ £ when T is clear from 
context. 

The denotational semantics of the complexity expression constructors describes the cost and 
potential of the corresponding target-language constructors. For example, consider the evaluation 
of t = rs (again, we phrase this discussion in terms of closed expressions for clarity): 

r^Xx.r 1 s^v' r 1 \x t-4 v'} 4- v 
rs I v 

cost(r) and cost (s) both contribute to cost(f). Recalling our earlier discussion of higher-type poten- 
tials, if ||r|| and are the complexities of r and s, then ||r|| (||5|| ) (a complexity) gives both the 
cost of evaluating r'{x i-> v'} as well as its potential; but its potential is also the potential of t. Thus 
we define the meaning of * expressions so that 

[IMI * Mil = (i + Illi J + IlkU + ([IkUaNlpDL ([IkMrtlMIJ)),) • 

We frequently need to "add cost to a complexity," so we define dally(«, (c,p)) = {n + c,p). Now we 
can write, for example, 

I||r|| * = dally (1 + I||r||J + [|k||,l, [llriydll.HJ)). 

pease and pf old expressions have a slightly more complex semantics that might be expected. 
Focusing on the former, the non-zero branch of a pease expression must take the maximum of 
the two branches, rather than just the second branch, even though a non-zero potential ought to 
correspond to a non-empty list. Because our goal is to establish upper bounds on complexity (hence 
potential), it may be that the branching expression in the target expression evaluates to nil, but its 
translation has a non-zero potential. As an example, consider the term t defined by 

case (if true then nil else [0]) of ([0,0], [x,xs]nil) 

The test expression translates to a complexity with potential 1 ; if \\\t ||] were to take into account only 
the non-nil branch, we would conclude that t has complexity (7,0) (cost 7, potential 0), whereas in 
fact it has cost 9 and size 2. A similar issue arises with pf old, although in this case the cost is not 
an issue; because pf old is a structural recursion, the base case expression will be evaluated, and 
hence its cost included in the total cost (see Lemma [5^61 for a precise statement). 

4. Translation from target to complexity language 

The translation from target language to complexity language is given in Figure [8] We assume a 
bijection between target and complexity variables, so that when we write ||jc|| = x, the occurrence 
of x on the left-hand side is a target variable and the occurrence of x on the right-hand side is the 
corresponding complexity variable. The translation of the simplest target expression constructors 
gives expressions that describe directly the cost and size of the corresponding target expressions; 
more complex constructors are translated to the corresponding complexity constructors, and we 
leave it to the denotational semantics to extract the complexities from there. There is a choice to 
be made regarding which constructors in the target language have corresponding constructors in the 
complexity language. Certainly fold must be in this list, so that recursive programs are mapped 
to recurrences. But whether abstraction and application should have counterparts, or whether they 
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e ::=X \ N | e + e \ e\l e \ (e,e) \e c \e p \ X^x.e \e*e 

pease e of (e, [p, ps]e) | pf old e of (e, [p,ps, w]e). 

Figure 4. Complexity language expressions. X is a set of variables and N a set of 
constants for each n G N. 

a,T::=Nxy y::=N|y— s>T 

Figure 5. Complexity and potential types. The full complexity types consist of the 
simple types with products over N. 



F,x:o\-x:o rh«:N 
Thr:N rKsjN Thr:T Ths:T 
r I /• • s : N ^' ' F /• , s : r 
rhr:N s:y r\-r:"p Fhr-.f 2 
rh(r,s):"p Thr c :N T h r p : y 
r,x:y°hr:77 D rhr:y^ n Tj n rhs:y° 
Th A*x.r:y^ n T] n rhr*j:i) D 
Thr:N Th^:^ r,p:N, j«:N h f : 7° 
r h pease r of (s, [p,ps]t) : y° 
Thr:N Fhs-.f 2 r,p:N,^:N,w:f hnf 
Tl-pfoldrof (5, [/7,/«,w]0:y n 

Figure 6. Complexity language typing rules. Recall that y° = N x y and y— » D T = 

(y^T) n 

should be translated into complexity expressions that directly describe the denotational semantics 
of A* and *, is not completely clear. The choice we have made seems to allow simpler reasoning 
about the translated expressions, reasoning that we should be able to easily formalize in Coq, and 
which would also be more familiar to programmers. 

Before proceeding to examples, we note that translation preserves type derivations. For a target 
type context F, define ||r|| = {(x: ||t||) | (x: t) G T}. 

Proposition 4.1. IfT hr:r, then \\T\\ h ||r|| : ||t||. 

4.1. Examples. We show the results of translating the insertion-sort and map functions in this 
section. We freely transform expressions in the complexity language according to validities in the 
semantics, for example adding natural numbers and computing maximums when possible. The 
soundness of each such equality or inequality is easily proved, and so such transformations could 
easily be incorporated in a formalized proof that simplifies into a more amenable form. 
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[r h pease r of (s,\p,ps]t):)P}% 



[TI-rVj:T]§ = [rKvH§ 

[rhM:^K = (HU^) 
[rhr c :N^ = ^ ([rM) 
[rhr p : 7 ^=^([rM) 
[rhA,x.r: 7 ^ D T] n M = (l,^.[r,x:^hr:T7 n ^{x^(l^)}) 
[r h r « : T] n M = dally(l + [r c ]§ + M£ , [r p ]§ ([j,]^) 

where ^ = £,{p,ps >->> l,q} 

[rhpfoldrof Cs.tp./w.w]*):? ]^ (2 + [rec c ]£> + , 

where rec = pf oldy of (5, [/?,/«, 

£0 = £{.y = €{p,/w,w h-> l,^r,(l,rec p )} 

Figure 7. Denotational semantics of complexity terms. 7Lp. ■ • ■ denotes the seman- 
tic function that maps p to • • • . 

4. 1. 1. Insertion-sort. We start by considering the list-insertion function defined by 

ins = Ax.Axs.fold xs of (x :: nil, 

[y, ys , w] if x <= y then x :: y :: ys else y :: w) 

and showing that it has a linear running time in the size of its input. Translating directly yields 

1 1 ins 1 1 = A*x,xs.pf oldxsp of ((2+x c , 1), 

\p,ps, w] dally (4 + x c , (4 + x c , 2 + ps) V (2 + w c , 1 + w p ) ) ) . 

If we write fi ns (x,xs) for ||ins|| *x*xs, apply the equations for the denotations of A* and *, and use 
properties of the order on natural numbers, we have 

f im (x,xs) = dally(4+x c +xy c ,pfoldxs ; , of ((3,1), 

\p,ps, w] dally (5 , (5 , 2 + ps) V (2 + w c , 1 + w p ) ) ) ) 

< dally(4+x c +xj c ,pf oldxs p of ((3, 1), 

\p,ps,w](l0 + w c , (2 + ps) V (l+w p ))) 

To turn this into a recognizable form, set 

g(z) = pf old z of ((3, 1), \p,ps,w](^ + w c , (2 + ps) V (1 + w p ))) 
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((b)) = N «<T -> T» = ((a)) -> ||T|| ||T|| = N X «T» = ((T)) C 



\\x 
\\c 
\\rRs 
||nil 
||r :: s 

||if r then 5 else ? 

||Ax.r 
||rj 

case r of (5, [x,xs]f) 



(1,1) 

(2+ \\r\\ c + ||j|| c ,1) 
(1,0) 

(l + llic+IMIci + lky 

dally(l + ||r|| c ,p|| V||?||) 
: A*x.||r|| 

||r|| * \\s\\ 
:daUy(l + ||r|| c , 

pease ||r|| p of (||j||, \p,ps] \\t\\{x,xs h-> (1,/w)})) 
||f old r of (5, [*,xy,w]f)|| = dally(l + ||r|| c , 

pfold ||f||p of (\\s\\,[p,ps,w}\\t\\{x,xs ^ (1,/?),(1, /«)})) 

Figure 8. Translation from target types and expressions to complexity types and 
expressions, c ranges over integer and boolean constants. 

so that fi ns (x,xs) = dally (4 + x c +xs c ,g(xs p )). Rewriting g as a pair of recurrences (one each for 
cost and potential) we obtain 

gc(0) = 3 g c (q+\) = \3 + g c (q) 

8p(0) = l g p (q+\) = (2 + q)y(\+g p (q)) 

Here again we have used the equations from the denotational semantics of pfold. For example, the 
expression for g c (q + 1) is 2 + rec c + t c where rec is the recursive call (i.e., g(q)) and 

t = (10 + w c ,(2 + /m)V(1+w p ))[wi-> (l,nc p )] = (11, {2 + ps) V (1 + rec p )). 

A straightforward induction establishes g(z) < (13z + 3,z+ 1) and hence f ins (x,xs) < (13xs p + 7 + 

I XS c - XS p I 1 ) • 

Continuing, we now consider the insertion-sort function defined by 
ins_sort = Axs.fold xs of (nil, [y, ys , w](insert y w)) 
Following the approach above and writing fi srt (xs) for ||ins_sort|| *xs, we have 
g{z) = pf oldz of ((1,0), [p,ps,w]f isrt ((\,p),w)) 
fisrt (xs) = dally (2 + xs c , g (xs p ) ) 
g c (0) = l g c (q+\)=2 + g c (q) + {f ins {(\,\),(\,g p (q)))) c 

g p (0)=0 g P (q+i) = {M(hi),(hgp(q)))) p 

Using our bound on fj ns , a proof by induction to show that g p (q) < q, and then this last inequality 
to simplify the bound on g c (q + 1) we have 

gc(0)<l g c (q+\)<U+g c (q) + \3q 

g P (0)<0 gp(q+l)<l+l 
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from which we conclude g(z) < (13z 2 + llz+ l,z) and hence fi sr t{xs) < (l3xs^+9xs p + 3+xs c ,xs p ). 

Although we don't often think of the analysis of insertion-sort involving an analysis of size 
(something we are forced to do when working with complexities), in fact such size analyses are 
usually implicit. For example, in the standard analysis of insertion-sort, we usually implicitly make 
use of the correctness of the algorithm to assert that the length of ins_sort(xy) is the same as the 
length of xs. 

It is also worth noting that the analysis of || ins_sort || only depends on properties of || insert || , 
namely that fj ns (x,xs) < (a-xs p +x c +xs c + b,xs p + 1). One way to see this is to note that we could 
have defined a general fold function 

list_fold = Xf , xs , a. fold xs of (a, [y, ys , w](f y w)) 

and analyzed ||list_f old||. The analysis would be in terms of f c and f p . We could then "plug 
in" assumptions about f c and f p to analyze concrete instances of the general fold function such 
as insertion-sort. Such assumptions and analyses could be exact or asymptotic, as the application 
demands. One can certainly imagine that such analyses are likely to be important when reasoning 
about large modular programs. 

4.1.2. Map. Perhaps surprisingly, the analysis of the higher-order map function defined by 
map = Ah.Axs.fold xs of (nil, [y, ys , w](h(y) :: w)) 

is more straightforward than for insertion-sort. Again following the approach above and writing 
fmap(h,xs) for ||map|| *h*xs, we have 

g(z) =pfoldzof ((l,0),\p,ps,w](4 + (h p (p)) c ,l+w p )) 

fmap (h,xs)= dally (4 + h c +xs c ,g(xs p )) 

gc(0) = 1 gc(q + l) = V + (h p (l)) c+gc (q) 

g P (0)=0 g p (q+\) = \+g p (q) 

The cost of applying h to any element of xs is fixed, because the size (potential) of every integer is 1 ; 
call this cost C. We conclude thatg(z) < ((7 + C)z+l,z) and hence fmap (h, xs) < ((7+C)xs p + 5 + 
h c -\~xs C jXS p j. 

5. Soundness of the translation 

We saw in our examples that the translations of the target-language programs yield expressions that, 
modulo some manipulations, describe the expected upper bounds on the complexities (and hence 
evaluation costs) of the original programs. One might worry that the manipulations themselves are 
the source of success, rather than the translation. Our main goal is to show that (in an appropriate 
sense), cost(f) < cost([||f||J) for all programs. The challenge in doing so is that cost (and potential) 
is not compositional: cost(rs) is not defined solely in terms of the cost of subexpressions of r and s. 
To get around this, we define a relation between target language closures and complexities, which 
we then g eneralize to ta rget and complexity expressions. The relation itself is essentially a logical 
relation [Mitchell 1 19961 Ch. 8] that allows us to prove our main Soundness result by induction 
on terms. With this in mind, we start by defining the bounding relation C as follows. Let t be a 
target-language expression, t, a value environment defined on the free variables of t, a be a target 
language type, and % be a complexity such that % S [|| G ||] . We define t E, Q a % if t E, \. v 6 implies 

• cost(f^) < cost(^); and 

. vdQ^pot(x). 
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The value bounding relation C relates values to potentials: 

• (tt)0 Sli 1, (ff)e Qlti hnO E™t I- 

• (n , • • -,n k -i)d C™^ pifk <p. 

• (Xx.r)Q Eff^T p if whenever ztj (Z™ 1 then r 0{x i-> 217} C T 

We will usually drop the type-subscript from C and C val . If £ and are value- and complexity- 
environments respectively, we write § C ^* to mean that xt; \Z a E,*{x) whenever x G Dom <§* and 
£*(x) G [He HI- If * is a target expression and P h f : T a complexity term, then we write ( CP h 
t* : x to mean that f § C whenever is a r*-consistent environment and £ C £*. 

Our main theorem is the following: 

Theorem 5.1 (Soundness). 7/T h t : z, then t C ||T h ? : t||. 

Corollary 5.2. If_\-t:r, then cost(7 {}) < cost(||_ h ? : t||). 

The reader may be concerned by the seeming lack of a connection between any putative value 
environment £ under which t may be evaluated and T; in particular, there is no connection between 
<^(x) and T(x) for x € fv(f). A short response is that the Soundness Theorem is typically applied 
only to closed terms (as in the Corollary) and hence there is no concern, and the reader is free to 
accept this and ignore the remainder of this paragraph. A longer response is that this observation has 
to do with the fact that our evaluation semantics places no restrictions upon the value environment to 
enforce reasonable "typing." Indeed, no restrictions can be placed, as we have not defined a notion of 
typing for values, although defining such a notion is not technically difficult. However, in applying 
the Soundness Theorem and chasing the definitions, one is forced to apply it to value environments 
that assign "reasonable" values to the variables. For suppose that r h t : T; t C ||T h t : t||; % is a 
value environment such that 1 £ Ivd; and that x G fv(?). Since x G fv(?), x G Dom F. Take any 
I |T| I -consistent environment £,* such that £, C £,*. Suppose that £,*(x) G [||a||]; by the definition of 
consistency, ||r||(x) = \\a\\ and hence T(x) = a. But in addition, sincex^ C CT i§*(x) andx§ l£(x), 
E™ 1 £*(•*)• Examining the definition of C™ 1 , we see that this ensures that the "type" of £(jc) 
(had we defined it) must be a. 

Before proving the Soundness Theorem, we establish some preliminary lemmas. The first three 
are consequences of definitions and syntactic manipulation. 

Lemma 5.3. Ift£, C T % then for all %' G [||t||]|, ?<§ 

Lemma 5.4. lf% C andvd C val q, then %{x^vd} C £*{x^ (1,<?)}. 

Lemma 5.5. [?]<^{xi-> (a,b)} = (a,j)}]]<^ i-> ft} where y is afresh variable. 

The next two lemmas establish bounds related to recursive definitions. 

Lemma 5.6. For all complexity expressions r, 

cost(Hn <cost([pfoldrof (s,[p,ps,w}tM*). 



Proof. Formally we prove by induction on q that for all complexity expressions r and environ- 
ments £,*, if {rj£,* = q, then 

cost(Mn <cost([pfoldrof (s,[p,ps,w}tM*)- 

If q = 0, then the two sides of the inequality are in fact equal. Suppose [r]£* = q + 1. Then 

cost([pf old r of (s, \p,ps,w]t)}^*) = 2 + cost(rec) + cost(lt}^* {p,ps,w i-» \ ,q, (1, pot (rec))}) 

where rec = [pfoldyof (s,[p,ps,w]t)}E,*{y q}. By the induction hypothesis, cost ([.?]<!;*) < 
cost(rec), and so the claim follows. □ 
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Lemma 5.7. Suppose s C \\s\\ and t C ||/[|. Fix E, C £,* and let E$ = <^{.y | — > («o ? • • • i n k-i)} an d 
= i— > q}, where k <q. Assume y is not free in either s or t. Then 

fold y of (s, [x,xs,w]t)Ej, C daUy(2, [pfoldy of (||j||, [p,/M,w]01§T) 

where t' = \\t \\{x,xs i-> 

Proof. We prove the claim by induction on t If k = then the cost of the fold expression is 
2 + cost (j£o)» an d the value of the fold expression under £ is the value to which s^o evaluates. The 
cost bound is proved by Lemma [5^6] 

cost(foldyof (s,[x,xs,w]t)Zo) < 2+ (||j||£ *) c < 2+ ([pf oldy of (\\s\\,\p,ps,w]t)jEJ) c 

By the IH vd C val (||j||<!joV The potential of dally (2, [pf old? of (\\s\\,[p,ps,w]t)}^) is either 
(II^H^o ) p or (II^H^o ) P V (p']^o*)p for some environment we know from Lemma 1531 that the 
potential bound holds. 

Suppose %o(y) = (n,ns) with ns = (no,... ,njt-i). Then E^{y) = q + 1 for some q > k. By the 
induction hypothesis we know that 

fold y of (s, [x,xs,w]t)E J {y h-> ns} C dally (2, rec) 

where r<?c = [pf old y of (||.s||, [p,/w,w]f)]£*{;y i— >• q}. So if 

(fold j of (s, [x,xs,w]t))^{y i-> ns} J. v 0' 

then v' 0' C val rec p , which means 

£{x,xs,w h-» 71,71s, v'0'} C i->- (1, 1), (1,^), (l,recp)} 

Let £j = ^q{x,xs,iv i-^n,nj,v / / } an d l et £i* = ^q{PjP s i w 1 — ^ 1 , <?, (l,rec p )}. Since y does not occur 
free in t or f', by Lemma [531 1 E J \ C [f']^*. So 

cost(f old j of (5, [x,xy,w]?)£o) = 2 + cost(f oldy of (5, [x,xy,w]f)£{y 1— > nj}) +cost(?<^i) 

<2 + (2 + rec c ) + (Pl^) c 

= 2 + ([pf old 3; of (|M|, [p,/w,w]0j£o) c 

For the potential bound, if (f oldj of (j, [jc,xy,vi/]/))£o I v0, then r^i | v0, so vd C val pot ([*']£*). 
By Lemma [531 we have 

vd C val (\\sUo*) p V ([f'l^Dp = pot ([pf oldy of (11*11, [p,/w,w]0]&) • 

□ 

Proof of Soundness Theore m. By induction on the derivation of T h r : T. Most of the proof is similar 
to the analogous proof in banner and Royeri l2009ll . so we do just a few cases here. In this proof 
we will write \\s\\£* for l\\s\\}%*. 

Suppose rh(r::i): int* and fix E, QE,*. We have that \\r :: s\\ = (1 + ||r|| c + ||j|| c , 1 + \\s\\ p ). 
For the cost bound, 

cost((r :: s)%) = 1 +cost(r<^) + cost(5(§) < l + cost(||r||^*) + cost(||j||^*) = cost(||r :: s\\^*). 

For the potential bound, if (r :: s)E, \. (n,no, . . . ,n/c-i), then sE, \. (no, . . . ,nk-i), so by the induction 
hypothesis, pot(||j||^*) = q for some q>k; hence pot(||r :: s\\Ej*) = 1 +q > 1 +k. 

Suppose r h Xx.r : o~ — > T, with F,x i-> a h r : T, and fix £ C £ *. Verifying the cost bound 
is trivial, so we focus on the potential bound. Set p = j|Ax.r|| p ^* = JJLv.||r||£*[jc h-» (1,v)]. We 
must show that if zf] E™ 1 q, then r^{x H> zT]} C T /?(g) = ||r||^*{jc i-> (l,q)}. Since zT] IZ val by 
Lemma [5~4l £ {x 1— > zf]} C £*{je i->- (l,g)}, and so the claim follows by the induction hypothesis 
for r. 
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Suppose r h rs : X, with r h r : a — > T and V h s : a, and fix t, C ^ *. By unraveling definitions, 

Hir=daity(i+(iir|ir) c +(iHi5%(ikiir)p(Nir)p)- 

Suppose {rs)£, \, v8 by the following evaluation rule: 

i^jw^ r'd l {x^wd 2 }ivd 

We first show that 

r'e l {x^wG 2 }Q(\\r\\^) p (\\s\\^) p . (*) 
Since r£ | (Ax/)0i, we know that (Xx.r J )6 1 != val pot||r||£*. Hence if ztj C val then r'e^x ^ 
zil} E (IM|5*)p(p). Since s£ C and s£ |w0 2 , we have that wfl 2 E val and (*) 

follows. 

To establish the cost bound, we compute 

cost((r.y)§ ) = 1 + cost(n§ ) + cost(s t, ) + cost(7 di [x H> wd 2 ]) 

< 1 + cost(||r||r) + cost(|Mm + cost((\\rU*) P (\m*) P )) 
= cost(||rj||£*) 

with the inequality following from the induction hypotheses and (*). The potential bound follows 
from (*) and the fact that r 1 Q\ [x \- > w d 2 ] i v . 

Suppose r h fold r of (s, [x,xs,w]t) : T and set t' = \\t\\{x,xs i-» (l,p), (l,ps)} so that 

||f old r of (s, [x,xs,w]t)\\ = pf old ||r|| p of (||j||, [p,ps,w]t) . 

Suppose Ins; we prove the claim by induction on ns. Suppose ns = ( ). We start by computing 
the cost: 

cost(f old r of (s, [x,xs,w]t)^) = 1 +cost(r§) + cost(5§) 

<i+(ikiir)c+(iMin c 

< 1 + (\\rU*) c + ([pf old IHI, of (\\ S \\,\p,p S ,w}t')W)c 

= cost(||f old r of (s, [x,xs,w]t)\\£,*). 
The second inequality is an equality if ||r|| = 0, or follows from Lemma 15^61 if ||r|| > 0. Turning 
to the potential bound, if (fold r of (s,[x,xs,w]t))£, lv6 then sE, |v0 as well and so v0 IZ val 
pot ( 1 1 s 1 1 1, *) . If || r 1 1 = 0, this suffices to verify the claim. If || r \ \ > 0, use Lemma 15.31 This finishes 
the case in which r E, I ( ). 

Suppose rtj \. (n,ns), where ns = (no, . . . ,«/t-i). By the induction hypothesis for r, \\r\\ p ^* = 
q + 1 for some q >k. Make the following definitions: 

• & = ${y ^ ns}; = %*{y^ q}. 

• red = f oldy of (s, [x,xs,w]t). 

• rec = [pfoldy of (\\s\\,[p,ps,w]t')j^. 

By Lemma|5771 rec' <§ C dally (2, rec). So if rec' <§ I v' G' then v' 6' E val rec p . Now set 

• i^i = ^{x,xs,w i— )• n,ns,v'6'}. 

• I* = €*{p>/">w->- !,?,(!, rec p )}. 
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By Lemma 1541 6 C £*, so by the induction hypothesis for t and Lemma [531 we have t E,\ C 
For the cost bound when r% \. (n,ns), we have 

cost (fold r of (s, [x,xs,iv]00 = 1 +cost(n§) + cost(r<?c f i§{;y i-> ?w}) + cost(? 

<i + (||ir)c+(2+«c c ) + (M^)c 

= 1 + (||r||^) c + cost([pf old ||r|| p of (||*||, [p,ps,w]t'M*) 
= cost(||f old r of (s, [x,xs,w]t)\\£*) 

For the potential bound, suppose (foldrof (s,[x,xs,w]t))% ^vd. Then we must have f <§i J, v as 
well; hence vd C val pot([/']^*). So by Q, 

v0 C val pot(||j||^*) Vpot(p'l^) = pot(||foldrof (j,[x,xs,w]0||£*). 

□ 



6. Implementation in Coq 

We have implemented a subset of the target and complexity languages in Coq that includes the 
simply typed A -calculus with integer and boolean operations. For this subset we have implemented 
the translation function and proof of the Soundness Theorem. The current development may be 
found at 



http : / / wesscholar . wesleyan . edu/compf acpub 
When complete, the formalization will provide a mechanism for certified upper bounds on the cost 
of programs in the target language. Since the denotational semantics is built on Coq's built-in type 
system, one can use the formalized Soundness Theorem to establish a bound on a given target 
program, then continue to reason in Coq to simplify the bound, for example establishing a closed 
form if desired. 

Although we use Coq to formalize the translation, our system is really a blend of external and 
internal verification. Our long-term goal is to be able to translate from programs written in a lan- 
guage such as SML or OCaml. Such progra ms would not have cost information directly associated 
with them via annotations (such as done bv iDanielssonl j2003h ') or a type system. In this sense, our 



approach follows that of CFML I Chargueraud . 2010tl . in which Caml source code is translated into 



a formula that can be used to verify post-conditions. 

The main non-trivial aspect of the development is the definition of the bounding relation. The 
bounding relation is a simultaneous recursive definition to two relations, C and C val . In Coq the 
difficulty arises in defining Q a -n: in terms of E™-^t ( m tum defined in terms of C T ), which is not a 
structural descent on type. We resolve this by defining subsidiary versions of C and C val that take 
a natural number argument, and which are structurally decreasing on that argument. C and C val are 
then defined in terms of these relations, using a numeric value sufficiently high that the inductive 
definition is guaranteed to terminate by reaching a base type, rather than a value of for the numeric 
argumentQ With this out of the way, the proof of the Soundness Theorem proceeds more-or-less as 
described in this paper. 



7 We also implemented a version in which C val is inlined into the definition of C as an anonymous fixpoint, but found 
that it became even more tedious to prove the required lemmas, because we often have to reason specifically about C val . 
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7. Related work 



The core idea of this paper is not new. There is a reasonably extensive literature over the last several 
decades on (semi-)automatically constructing res ource bounds fr om source code. The first work 
naturally concerns itself with first-order programs. Wegbreij | 1975 1 describes a system for analyzing 



simple Lisp programs that produces closed forms that bound running time. An interesting aspect of 
this system is that it is possible to describe probability distributions on the input domain (e.g., the 
probability that the head of an input lis t will b e some specified value), and the generated bounds 
incorporate this information. iRosendahl [1989] proposes a system based on step-counting functions 
and a bstract interpretati on for a first-order subset of Lisp. More recently the COSTA project (see, 



e.g.. [Albert et all 12007]) has focused on automatically computing resource bounds for imperative 
lang uages (actua ll y, byte code). 

Le Metayer 's lll988ll ACE system is a two-stage system that first converts FF^ programs to re- 



cursive FP programs describing the number of recursive calls of the target program, then attempts 
to transform the result using various program-transformation techniques to obtain a closed form. 
Jost et all kOldl describe a formalism for automatically infering linear re source bounds on higher- 



order programs, provided of course that such bounds are correct, and iHoffmann and Hofmannl 



1 2010] extend this work to handle polynomial bounds. This system involves a type system analogous 
to the target-language system, but in which the types are annotated with variables corresponding to 
resource usage. Type inference in the annotated system comes down to solving a set of constraints 
among these variables. 

Here let us delve a little more into the systems mentioned in the introduction, and jus t those 



aspects concerned with a general analysis of higher-type call-by-value languages. Shultis [1985] 



defines a denotational semantics for a simple higher-order language that models both the value and 
the cost of an expression. As a part of the cost model, he develops a system of "tolls," which play a 
role similar to the potentials we define in our work. The tolls and the semanti cs are not use d directly 
in calculations, but rather as components in a logic for reasoning about them. IS an H |l990] puts for- 



ward a translation scheme in which programs in a target language are translated into programs in the 
same language that incorporate cost information; several target languages are discussed, including 
a higher-order call-by-value language. Each identifier / in the target language is associated to a cost 
closure that incorporates information about the value / takes on its arguments; the cost of apply- 
ing / to arguments; and arity. Cost closures are intended to address the same issue our h igher-type 
potent ials do: recording information about the future cost of a partially-applied function. IVan Stona 



|2003] annotates the operational semantics for a higher-order language with cost information. She 
then defines a category-theoretic denotational semantics that uses "cost structures" (which are re- 
lated to monads) to captu re cost information and shows that the latter is sound with respect to the 



former. iBenzinger [2004] annotates NuPRLs call-by-name operational semantics with complexity 



estimates. The language for the annotations is left somewhat open so as to allow greater flexibil- 
ity. The analysis of the costs is then completed using a combination of NuPRLs proof generation 
and Mathematica. Benzinger's is the only system to explicitly involve automated theorem proving, 
though Sand's could also do so. 

These last formalisms are closest to ours in approach, but differ from ours in a key respect: the 
cost domain incorporates information about values in the target domain so as to provide exact costs, 
whereas our approach focuses on upper bounds on costs in terms of input size. We are hopeful that 
our system proves amenable to analyzing complex programs, but there is much work yet to be done. 



8 I.e., Backus' language FP lBackud Il978ll . 
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8. Conclusions and further work 

We have described a static complexity analysis for a higher-order language with structural list recur- 
sion that yields an upper bound on the evaluation cost of any typeable program in the target language. 
It proceeds by translating each target-language program t into a program ||f || in a complexity lan- 
guage. We prove a Soundness Theorem for the translation that has as a consequence that the cost 
component of ||?|| is an upper bound on the evaluation cost of t. By formalizing the translation and 
proof of the Soundness Theorem in Coq, we obtain a machine-checkable certification of that upper 
bound on evaluation cost. 

The language described here supports only structural recursion on lists; an obvious extension 
would be to handle general recursion. This should be straightforward if we require the user to supply 
a proof of termination of the program to be analyzed. However, it should be p ossible to define 
the op erational semantics of the target language co-inductively (as done by, e.g., iLeroy and Gralll 
[2009]), thereby allowing explicitly for non-terminating computations. The complexity language 
semantics would then have to be adapted so that the denotation of a recursive complexity function 
may be partial; t he fou ndat ions for such denot ational semantics have already been carried out by 



Paulin-Mohrin g [2009] and Be nton et al. [2009]. Indeed, one could then hope to prove termination 
by extracting complexity bounds and then proving that these bounds in fact define total functions. 

Although we have adapted our formalism to a few other inductively-defined datatypes (e.g., 
binary trees), we suspect that there are hidden difficulties in generalizing the system to handle ar- 
bitrary inductive definitions. One such difficulty might be, e.g., binary trees in which the external 
nodes are labeled by values of another inductively-defmed datatype. If we wish to consider both 
trees and labels as contributing to size (potential), then it seems that only external nodes labeled by 
size-0 values have potential 0. T hat in turn may m ake it difficult to develop useful cost bounds for 
functions that ignore the labels. Jost et all I 20ldl deal with this issue by annotating the type con- 



structors with resource information, and hence automatically account for the resource information 
for all types of objects reachable from the root of a given value. We have investigated a similar 
approach in our setting, in which potential types mirror the target language types. For example, 
we would define ((a list)} to be essentially ((a)) list. This becomes rather burdensome in prac- 
tice, and a mechanism for minimizing the overhead when the generality is not desired would be a 
necessity. 

Another obvious direction would be to handle different evaluation strategies and notions of cost. 
Compositionality is a thorny issue when considering call-by-need evaluati on and l azy da t atypes, and 
it may be th at amortized cost i s at least as interesting as worst-case cost. Sands! 1 1990ll . lVan Stond 



ll2003ll, and rPanielssonl §003] address laziness in their work. The call-by-push-value paradigm 



IILevyL Il999ll gives an alternative perspective on our complexity analysis. Call-by-push-value disin 



guishes values from computations in a monadic-like approach under the maxim "a value is, a compu- 
tation does." With this in mind, we might adopt the following statement with respect to complexities: 
"potential measures what is, cost measures what happens." An alternative presentation of our work 
might utilize a call-by -push- value target language to emphasize the distinction between computation 
expressions and value expressions and what those mean for the complexity analysis. 

The development in Coq currently works directly with the expressions of the form ||?||. These 
are moderately messy, as can be seen from the examples. It would be nice to provide a more 
elegant presentation of these complexities along the lines as the discussion of the examples. In the 
same vein, it would be very useful to develop tactics that allow users to transform comple xities into 
simpler ones, a ll the while ensurin g the appropriate bounds still hold. iBenzingen 1200411 addresses 



this idea, as do lAlbert et all 11201 lH of the COSTA project. Another relevant aspect of the COSTA 
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work is that their cost relations use non-determinism where we have used a maximization operation 
to handle conditional constructs; it would be very interesting to see how their approaches play out in 
our context. These tactics could be then applied manually or automatically according to the user's 
preferences using Coq's built in automation tools. Ultimately we should have a library of tactics for 
transforming the recurrences produced by the translation function to closed (possibly asymptotic) 
forms when possible. 

Acknowledgments. We would like to thank the referees for several helpful suggestions including 
pointers to additional related work. 
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